Vision Research

Face recognition ability varies enormously across humans. Individuals with face-blindness (prosopagnosia) struggle to recognize even close family members while super-recognizers can identify strangers with exceptional accuracy. Where do ANNs fall within the human face recognition spectrum? Here, we administered the standardized tests used to characterize human face recognition ability to a diverse set of models, allowing us to contextualize a model's performance within the distribution of human behavior. We found that the majority 55% of models to be classified as face-blind and that even the best face-trained models do not cross the human threshold to be considered a super-recognizer. Interrogating the internal representations of these models showed that models that performed well on standardized face recognition tasks were more identity-selective and viewpoint-invariant. We also found that low-performing models did contain some identity information in independent representational subspaces. Removing viewpoint-dependent subspace improved face recognition abilities in 49 of the 53 models tested. Targeted unit ablations further identified opposing contributions, with viewpoint-dependent units disrupting identity coding and viewpoint-tolerant units supporting it. Together, our results show that most AI models are face-blind with worse face recognition ability than humans, and demonstrate how differences across AI models can be used generate testable hypotheses about the computational basis of human face recognition in humans which can then be probed in future studies.

Changes in pupil size have been widely used as an indirect measure of attention. However, attention engages on a faster timescale than the pupillary response. Therefore, the slow time course of the pupil response makes it difficult to distinguish attentional responses to stimuli presented in close temporal succession. Although signal deconvolution has been used in previous research to estimate latent attentional responses from the pupil signal, existing methods primarily model dilation without considering the visual stimulus evoked constriction phase of the pupil. Here, we introduce a dual-kernel deconvolution model that models constriction and dilation using separate impulse response functions. Using an RSVP paradigm with emotional faces as target stimuli, we demonstrate that the proposed model estimates latent attentional response despite substantial overlap in the measured pupil signal. Importantly, the recovered attentional response resembles the time course of attentional allocation previously predicted by several models of Attentional Blink. Together, these findings demonstrate that the dynamics of latent attentional episodes can be recovered from the pupil response with a temporal resolution previously obscured by the sluggishness of the pupillary response. The proposed method is also provided as a usable software package that serves as a general tool for investigating the temporal dynamics of attention in rapid visual paradigms.

In fully-sighted individuals, pursuit and saccadic eye movements work together to foveate moving targets; pursuit attempts to match target velocity, while catch-up saccades re-align the fovea to the target if it deviates during pursuit. Although saccade-pursuit interactions are well understood in healthy individuals, much less is known about how these mechanisms adapt in the presence of central vision loss. Here, we simulated central vision loss in healthy young adults to examine plasticity in saccade-pursuit mechanisms and investigated whether and how catch-up saccades adapted over five days of visual tracking. Twenty-five fully sighted participants performed a task in which they pursued a moving target that stepped position randomly (designed to elicit catch-up saccades) while viewing with a gaze-contingent central scotoma. The scotoma was shaped like a bitten cookie to encourage use of a specific preferred retinal locus (PRL) corresponding to the peripheral location with the highest acuity. By the end of the five days of repetition, most participants showed catch-up saccade endpoints shifted towards the PRL rather than the fovea. On average, endpoints shifted 0.5 deg towards the PRL between days 1 and 5 (range=0.3-1.5 deg), with significant shifts in 13 of the 25 participants. Adaptation was asymmetric; shifts were largest when the target moved in the direction opposite the scotoma bite and smallest when it moved towards it. In summary, we observed adaptation of saccade-pursuit interactive mechanisms through the re-referencing of catch-up saccades from the fovea to PRL. These results provide a promising avenue for rehabilitation in central vision loss aimed at improving eye movements and encouraging PRL use. Ultimately, such an approach may enhance residual visual function and quality of life.

Attentional selection not only increases behavioural performance and neural information for attended stimuli but, in some instances, induces changes in stimulus appearance. While reported appearance changes have mostly concerned the effects of attending to a particular spatial location, here we report a shift in appearance that occurs when attention is directed to one component of a stimulus rapidly alternating at a single spatial location. Our stimuli included two frames of different colors, which alternated at 3.75Hz or 7.5Hz. Participants reported the perceived dominance of one frame's color in the reference stimulus, by comparison with a target stimulus. We found that when one stimulus frame was more salient than the other, with a greater likelihood of capturing exogenous attention, there was an illusory perception that the frame was physically present for a greater proportion of time. When the frames were matched in salience, a task encouraging participants to endogenously direct their attention to one frame produced a smaller shift in appearance. Our results add to the body of literature demonstrating that attention can change appearance, in this case altering the appearance of attended versus unattended stimuli that are presented at the same spatial location. Furthermore, our results demonstrate that at the 'temporal transparent' alternation rate of 7.5Hz, where rapidly alternating stimuli are perceived as two translucent surfaces, attentional selection of one surface can account for the apparent paradox that feature conjunctions are perceived at stimulus alternation rates higher than predicted by the temporal limits of feature binding for other stimuli.

Efficient motion perception requires integrating immediate sensory evidence with internal representations shaped by experience. This influence of recent history can produce serial dependence effects (SDEs), in which current judgments are biased toward or away from the source of previous information. We examined SDEs in auditory, visual, and cued audiovisual motion-direction discrimination using aggregate psychometric measures and trial-level generalized linear mixed-effects models (GLMMs), which estimated the independent contributions of the previous stimulus direction and the participant's previous response. Participants completed unisensory (n=57) and cued audiovisual (n=48) motion-discrimination tasks. Aggregate analyses indicated attractive SDEs in auditory conditions, repulsive SDEs in visual conditions, and no net SDE in the audiovisual-cued condition. Trial-level modeling revealed a more nuanced structure: previous stimulus direction produced repulsive effects in both unisensory conditions, with previous visual direction influencing judgments across all conditions containing visual motion, whereas previous response produced attractive effects only when visual information was task-relevant. In the audiovisual-cued condition, the absence of a net psychometric shift was consistent with the opposing influences of an attractive response-history effect and a repulsive visual stimulus-history effect rather than an absence of serial dependence. These findings suggest that serial dependence in motion perception reflects dissociable and concurrently operating processes, including a predominantly visual, adaptation-like stimulus-history effect and a task-dependent, decision-related response-history effect.

Age is a fundamental social attribute that affects rights, responsibilities and even social status. Color information of the face can affect perceived age, but the nature of its contribution, independent of luminance information, remains underexplored. To better characterize the role of color in age perception, we conducted two experiments. In our first experiment, we tested whether natural age-related changes in facial color alone can drive changes in perceived age. In our second experiment, we manipulated red-green contrast across faces aged 18-91 and tested the effect of this manipulation of perceived youthfulness. Our findings demonstrate that natural age-related color changes do provide cues for age perception, dissociable from luminance information. We also found that the optimal red-green contrast level to make faces look the youngest was significantly higher than the originals, meaning that faces tended to appear younger when their redness was accentuated, even though average face redness increases with age. Our analysis revealed that younger faces required greater increases in redness level compared to their older counterparts to achieve the youngest appearance. These results suggest that the spatial distribution of redness contributes to apparent youthfulness, especially in younger faces.

In camouflage research, field studies provide ecological validity but lack experimental control, and computer-based experimental studies have the opposite issue. Virtual reality (VR) systems could potentially allow researchers to have both experimental control and ecological validity. Here we develop a VR environment using Unreal Engine and present it via a head-mounted display (HMD). The HMD affords binocular depth and therefore a more ecologically valid, three-dimensional environment, whilst still maintaining experimental control. Disruptive camouflage interferes with depth-based cues, making it a good candidate for testing within a VR environment. Disruptively camouflaged moth-like targets were presented to human surrogate predators, and their survivability was measured. We show that VR can be used to measure camouflage effectiveness by replicating existing findings showing that edge-enhanced disruptive camouflage (EE-DC) is more effective than disruptive camouflage (DC) alone. Furthermore, we extend these findings to show that the effectiveness of EE-DC is modulated by the relative contrast of the disruptive patches. Specifically, it is more effective when the contrast difference between the patches is low. Additionally, we considered edge-enhancement in isolation and found that it still provided a camouflage benefit even without a DC component. Search paradigms within an immersive VR experimental framework offer a unique opportunity to evaluate camouflage theory in a rigorous, yet ecologically grounded manner.

The retina encodes visual information through parallel pathways formed by distinct retinal ganglion (RGC) types. Current understanding of how each cell type contributes to downstream visual processing and perception is limited by a lack of tools for selectively manipulating the activities of individual RGC types. Here, we introduce a computational framework called STORM (SpatioTemporal Orthogonal Response Modulation) that creates visual stimuli that aim to selectively modulate the activities of specific RGC types while minimally affecting the activities of nearby RGCs of non-targeted types. To illustrate the approach, demonstrate its flexibility, and identify conditions under which each simulated RGC type could be selectively manipulated, we first applied this framework to simplified models of RGC function, such as Linear-Nonlinear-Poisson (LNP) and LN-LN cascade models. We also deployed this approach on more complex (and more accurate) models of RGC responses, based on convolutional neural networks. We demonstrate that differences in spatial receptive fields, temporal dynamics, response polarities, and nonlinearities can provide sufficient degrees of freedom for designing stimuli that selectively manipulate individual RGC types. We also generalize this approach to conditions where the receptive field locations are unknown for in vivo and clinical applications.

Serial dependence, whereby current perception is systematically biased toward the most recent sensory experience, has been demonstrated across multiple sensory modalities. However, whether serial dependence occurs during active haptic perception remains unknown. Moreover, because haptic perception of object geometry depends on hand movements for sensory acquisition (i.e., active touch), it remains unclear whether serial dependence, if any, originates from perceptual processing, movement, or both. Here, we investigated whether active haptic perception exhibits attractive serial dependence using a newly developed haptic device that let participants explore virtual slopes of different heights. Participants reported perceived slope heights, and we recorded their kinematics. Perceptual judgments were systematically attracted toward the most recently experienced slope, demonstrating serial dependence during active haptic perception. By contrast, none of the kinematic parameters exhibited corresponding serial dependence. These findings show that serial dependence extends to the somatosensory perceptual domain and suggest that the observed perceptual bias was not mediated by systematic changes in hand movements.

When objects are embedded in a complex lighting environment, the illumination they receive depends on their geometric relationship to the environment. We measured performance in an odd-one-out color constancy task, with objects rendered in complex lighting environments. Observers identified the odd object among four objects, with two rendered under each of two lighting environments, sampled from a wide range of object orientations and camera positions. Constancy was high. The presence of a surrounding context increased accuracy. Gloss provided no measurable benefit. We also tested a condition in which the global color impression provided misleading information about the lighting environment, but this impaired performance only slightly. These findings suggest that human color constancy remains robust even under complex lighting environments, contrary to predictions of many simple models of illuminant discounting.

We investigated how auditory and visual presentation of small numerical magnitudes and non- numerical magnitudes are passively processed at the neural level. Earlier research suggests automatic processing of quantities occurs rapidly in the brain. However, there are no studies on passive, pre-attentive change detection of magnitudes utilizing mismatch negativity (MMN) across sensory modalities in same participants, important to understand amodal processing of magnitudes. We recorded EEG in passive auditory and visual multi-feature oddball and control paradigms from the same adult individuals (N=104/105, respectively). In the auditory paradigm, the standard stimulus was a 2-part sound, while the deviant stimuli varied in frequency, intensity, gap length, and number of sounds (1-part and 3-part stimuli). In the visual paradigm, the standard stimulus was a dot array containing two medium-sized dots, while the deviant stimuli varied in numerosity (1 or 3 dots) and dot size (small or large). The results for the auditory paradigm showed significant MMN across all features, including comparisons with deviant-alone control stimuli, except for the 2-part vs. 1-part contrast. The results for the visual paradigm suggested differences in processing across all features apart from large and medium-size stimuli. These results replicate earlier MMN findings that the brain distinguishes pre-attentively changes in sound frequency, amplitude and gap. Auditorily presented small numerical magnitudes also showed pre-attentive processing but less robustly to the contrast 2 vs. 1. Visual pre-attentive processing of small numerical magnitudes was also observed. Together these findings could be interpreted to support the existence of abstract, pre- attentive processing of numerical magnitudes. Highlights* Auditory and visual processing of numerical and non-numerical magnitudes was examined with EEG * Auditory non-numerical magnitudes and medium vs. small dots replicated earlier MMN studies * aMMN and vMMN showed pre-attentive processing of changes in small numerosities 2 vs. 3 and 2 vs. 1 * 2 vs. 1 tone processing was less robust than 2 vs. 3 or the processing of the visual contrasts * The brain processes changes in small non-symbolic magnitudes pre-attentively

Estimating receptive fields in large neural populations is often limited by experimental time: stimuli with theoretically favorable statistics for receptive-field estimation may, in practice, evoke too few informative spikes in neurons with high specificity for structured features, requiring impracticably long recordings. Therefore, we asked whether a structured stimulus could broadly engage neural populations across stages of the early visual system while permitting a tractable and interpretable reverse-correlation analysis. To address this question, we developed "reverse correlation against stimulus elements (RCASE)", an analysis framework for stimuli that can be represented frame by frame through the presence or absence of specific stimulus elements. We then designed such a stimulus consisting of random moving objects (RMO) and compared RMO/RCASE with conventional spike-triggered averaging using a dense binary white noise (WN/STA) stimulus across different stages of the early visual system of the mouse, namely the retina, the nucleus of the optic tract, the superior colliculus and the primary visual cortex, and in the primate retina. To distinguish the contributions of spatial and temporal scales, motion, stimulus structure, and analysis choice, we additionally compared against WN/STA with different square sizes and frame rates, spike-triggered covariance analysis, as well as random static objects and spatially correlated cloud stimuli. RMO evoked stronger and more reliable responses than WN in the mouse retina and across the investigated mouse visual areas. RMO/RCASE yielded discernible receptive fields for a substantially larger proportion of neurons and in a fraction of the recording time. Although the stimulus objects featured localized moving contrast edges, this advantage was not restricted to direction-selective cell populations. In the primate retina, in contrast, WN/STA approached the performance of RMO/RCASE. RMO/RCASE also supported simultaneous estimation of stimulus direction, speed, and color tuning. Our approach enables rapid and interpretable functional characterization of large populations of visual neurons, enabling to simultaneously recover receptive fields and feature tuning, whenever a low-dimensional feature space can be specified beforehand.

Eye movements have been shown to reveal the cognitive processes underlying spatial ability, but most studies investigating gaze behaviour in spatial cognition use a single, screen-based task. Therefore, it is unknown whether these gaze behaviours are preserved across naturalistic task environments, or whether their relationship with spatial performance is domain-general or task-specific. The present study tested whether established gaze behaviours are preserved and generalizable across diverse, naturalistic spatial tasks. N = 30 young adults completed three naturalistic spatial tasks varying in perceptual, cognitive, and visuomotor demands: Q-bitz, the Brick Building Task (BBT), and the Mental Rotation Test (MRT). Gaze was measured using the Pupil Labs Neon eye tracker. Within-task analyses were conducted to assess whether gaze behaviours were preserved, and cross-task analyses were conducted to assess the generalizability of preserved gaze behaviours. Within-task analyses identified source inspection and transition rate as gaze behaviours that were consistently sensitive to changing task demands. Cross-task analyses showed that source fixation duration was a domain-general predictor of performance, whereas the relationship between transition rate and performance differed by task. Exploratory sex-separated analyses also suggested a more consistent gaze-performance relationship in males. These findings show that source fixation duration may represent a common cognitive process underlying performance across naturalistic spatial tasks.

Three recent publications have reported selective fMRI responses to images of food in the human ventral visual pathway. However, all three studies were based primarily on the Natural Scenes Dataset (NSD), in which high-level categories like food are correlated with other image properties such as the color and size of objects, and the distance of the scene. To test whether the reported food selectivity might reflect these or other correlates of food images rather than (or in addition to) food, we constructed novel stimuli that manipulated these properties on both food and non-food images, and collected data from new subjects in a data-rich design across two experiments pre-registered in OSF. We used a localizer paradigm based on a subset of NSD images to infer the "food component" in new subjects and then measured responses of this component to our new stimuli. In Experiment 1, we found that the food component response magnitude i) had a higher response to color than to greyscale images but showed no interaction between food and color, ii) showed a reduced preference for food over non-food when both were within reaching distance, and most importantly iii) was no higher for food than non-food when both were visually matched and presented as Cutouts on a white background. In Experiment 2, we found that the response of the food component could not be explained by object distance, real-world size, or mid-level visual image statistics. However, the response to non-food images with "gooey" material properties was as high as the response to food. Across both experiments, we consistently found a very low response (at or below fixation baseline) to NSD non-food images, which were predominantly outdoor scenes. Together, our results prompt a revision of prior claims including our own, indicating that the previously reported food component is better characterized as food-biased rather than strictly food-selective, and is driven in part by contextual or material features that are also present in non-food images. Our findings further highlight the importance of supplementing studies based on naturalistic images with experiments that unconfound image properties with carefully designed stimuli.

The control of visually guided bimanual movements is thought to represent a compromise between the drive for synchrony that characterizes many bimanual movements and the competing demands that characterize the constituent unimanual movements. As a result, while visually guided bimanual movements are found to be ostensibly synchronized, they also demonstrate subtle asynchronies. Here, we conducted a fine-grained analysis of bimanual reaching movements focusing on their relative synchrony as movements unfold and the relationship with gaze behavior. Our novel analysis revealed that participants frequently shifted their gaze laterally several times during bimanual reaches and that these gaze shifts were systematically associated with changes in the speed ratio of the two limbs. More specifically, we observed a strong association between gaze shifts and the speed ratio shifting toward the limb on the attended side. This pattern of behavior was seen regardless of hand dominance and target difficulty, suggesting that it may reflect a common visuomotor strategy during bimanual reaching.

Primates use eye movements to explore and gather information about their external, visual world. Similarly, even when there is nothing to see spontaneous eye movements reflect the structure and the exploration of internal, conceptual knowledge. However, it remains unclear what mechanism links eye movements to this internal search process. One possibility is that eye movements reflect visual imagery or visual exploration strategies used to explore conceptual knowledge. On the other hand, eye movements may reflect attentional movements in the conceptual space, which are independent from visual experience. Early blind individuals are a useful testbed for this hypothesis in that they never had access to functional vision, thus ruling out the influence of imagery or recycling of visual exploration strategies. To this end, we asked early blind individuals to perform a random number generation task while recording their eye movements. Using electrooculography (EOG), we found that eye movements track the exploration of the mental number line in both sighted and blind: participants moved their eyes to the right for positive changes in magnitude in the number sequence and to the left for negative changes, with eye movement amplitude scaling with the magnitude difference. Furthermore, sighted and early blind participants showed a similar temporal profile, with the effect peaking around 750 ms before number onset. These findings support the hypothesis that internally generated shifts of attention, rather than eye movements per se, drive the exploration of conceptual spaces.

Unveiling the plastic mechanisms of the primary visual cortex (V1) is crucial for understanding how visual processing can change and adapt, ultimately informing interventions to promote functional recovery following visual system damage. Paired associative stimulation (PAS) protocols can induce associative plasticity within the human V1. However, previous paradigms used full-field visual stimulation, engaging both hemispheres simultaneously, leaving unclear whether plasticity can be selectively targeted to a single visual pathway and whether it can modulate the non-stimulated hemisphere. Across three experiments in healthy participants, we investigated the neurophysiological, behavioural, and interhemispheric outcomes of a novel lateralized visual PAS protocol (vPAS). Pattern-reversal checkerboards were presented in a single visual hemifield and time-locked to transcranial magnetic stimulation (TMS) pulses delivered over the contralateral V1. Experiment 1 assessed neurophysiological effects by recording hemifield-lateralized visual-evoked potentials (VEPs) and testing three inter-stimulus intervals (ISIs: 75, 100, and 125 ms) between vPAS-paired stimuli. Experiment 2 evaluated perceptual changes in contrast detection, while Experiment 3 examined vPAS hemispheric specificity and aftereffects on interhemispheric visuomotor transfer. Only the 75-ms ISI consistently modulated visual processing in the non-conditioned hemisphere, reducing N75 amplitude and improving contrast detection for visual stimuli presented contralateral to the TMS side. Longer ISIs were not effective. Furthermore, this protocol altered interhemispheric transfer exclusively when the left, but not the right, V1 was targeted during vPAS. In conclusion, lateralized vPAS induces timing-dependent changes beyond the stimulated cortex, suggesting that associative plasticity can engage contralateral visual circuits and modulate visual processing.

Perception combines incoming sensory evidence with priors learned from preceding input. One example is the central tendency effect in which stimulus estimates regress towards a prior based on the mean of the stimulus distribution. Whether such priors are specific to each modality or shared across modalities remains unresolved. Here, we examined this question using speed perception in the visual, auditory and tactile modalities and used central tendency as a behavioral probe. By giving each modality its own speed range, clear predictions emerge when modalities are interleaved: if priors are modality specific, each modality should maintain its own central tendency bias whereas a supramodal prior would exhibit a different mean based on the pooled set of speeds. Moreover, a supra-modal prior should exhibit precision weighting, with the more reliable cue receiving more weight. For vision and touch, speed estimates converged when interleaved: both modalities shifted but the less reliable one shifted more, consistent with a precision-weighted supramodal prior. For vision and audition, the priors remained largely distinct: audition (the less reliable modality) showed a partial shift but did not converge on the precision weighted, supramodal mean. This was not explained by the large audiovisual reliability difference: it persisted when visual precision was reduced. These results show that in the motion domain, visual and tactile signals can combine crossmodally, but auditory and visual signals do not. Supramodal processing of motion therefore depends on which senses are involved and is likely explained by the lack of an early spatial mapping in audition.

Numerous studies have been conducted to investigate touch on the hand and forearm, but less is known about the rest of the body, particularly the torso and breasts, which play an important role in affective touch. We conducted psychophysical tests on three skin sites on the torso: upper breast, nipple-areola complex (NAC), and under the breast, as well as on two control sites: palm and forearm. We studied touch pleasantness and intensity by applying gentle caress-like strokes using a soft brush, and measured temperature and tactile detection thresholds. The results show an inverted U-shaped relationship for pleasantness over increasing stroking velocity for all skin sites, although we found some differences between sites, particularly at 30 cm.s-1, where participants found stroking on the NAC to be more intense and women additionally found it to be less pleasant. For temperature detection, we found that the NAC was more sensitive in men. For tactile detection, we mainly observed that the areola was far less sensitive than the other areas. Together, these results provide insights into tactile perception in the torso on a fundamental level, which can be applied to understanding how to regain sensations after mastectomy.

We explored the role of proprioceptive feedback from Ia afferents in intra-muscle synergies stabilizing muscle force with the help of muscle vibration, which is known to increase presynaptic inhibition of those afferents to alpha-motoneurons. Healthy young adults performed an accurate three-finger force production task by pressing with the index, middle and ring fingers of the dominant (right) hand. This was done without vibration and with vibration applied to the extrinsic finger flexors, extensors, both muscles, the palm, or biceps brachii. High-density electromyography was used to identify motor units in both finger flexors and extensors. The raster plots of motor units were converted into firing frequency functions and then subjected to principal component analysis to extract stable motor unit groups (MU-modes). The uncontrolled manifold (UCM) analysis was used to quantify the variance components affecting and not affecting force, and the synergy index in the MU-mode space. Contrary to our expectations, the synergy index did not drop under any vibration condition and showed an increase during vibration applied simultaneously to both flexor and extensor muscles. This was primarily associated with a drop in the component of variance affecting the force magnitude. The results speak against an important role of monosynaptic Ia-afferent projections to alpha-motoneurons in the formation of intra-muscle synergies. They suggest that muscle vibration can play the role of a noisy signal decreasing performance-affecting variance via stochastic resonance effects.